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A commentary on 

Is ego depletion too incredible? Evidence 
for the overestimation of the depletion 
effect 

by Carter, E., and McCullough, M. (2013). 
Behav. Brain Sci. 36, 683-684. doi: 10.1017/ 
S0140525X13000952 

The "strength" model conceptualizes self- 
control as a limited resource (Baumeister 
et al., 1998). Individuals are able to 
exert self-control, but only for a lim- 
ited period after which capacity declines 
leading to reduced self-control capac- 
ity; a state known as ego-depletion. The 
model has generated a sizable literature 
confirming the ego-depletion effect in 
multiple spheres. Our meta-analysis of 
published ego-depletion studies computed 
a medium-sized effect (d = 0.62) across 
198 tests (Hagger et al, 2010). 

Carter and McCullough (2013) recently 
applied analyses aimed at testing for 
publication bias to our data includ- 
ing Schimmack's (2012) incredibility 
index and two regression techniques 
(Egger et al, 1997; Moreno et al, 2009). 
Regression analyses indicated that the 
ego-depletion effect was substantially 
smaller than reported in our analysis and 
may even be zero, and the incredibility 
index indicated low statistical power and 
the chances of finding so many signifi- 
cant effects improbable. They concluded 
that the ego-depletion effect is subject 
to considerable publication bias and 
questioned whether it is a "real" effect 
at all. We replicated these analyses and 
found similar results. We have made our 



analyses available to download from the 
open-access Dryad Digital Repository 
(Hagger and Chatzisarantis, 2014). 

We thank Carter and McCullough 
(2013) raising the issue of bias. We 
take this opportunity to present some 
alternative conclusions to the ones they 
presented. We agree that journal editors 
should be more judicious in demanding 
bias tests in meta-analyses, but believe that 
that recommendation does not resolve 
the problem of interpreting the bias. 
An important addendum to the regres- 
sion analyses is that the bias detected 
by a significant regression line cannot 
be definitively attributed to publication 
bias. Sterne et al. (2000, 2001) suggest 
that such bias could be attributed to a 
number of possible sources. Instead, they 
use the term "small study" effect; the 
tendency for smaller studies to report 
larger effect sizes. One possible reason 
would be due to publication bias: jour- 
nals tending to favor the publication 
of small studies with statistically signifi- 
cant results and disproportionately large 
effect sizes. However, the findings may 
also be due to methodological inadequa- 
cies or true heterogeneity in the effect. 
A definitive response to resolving the 
nature of bias detected by these meth- 
ods (i.e., whether it is publication bias 
or other source of bias that causes a 
"small study effect") would be to demand 
authors conducting meta-analyses be dili- 
gent in the pursuit of "fugitive literature": 
unpublished studies with null findings, or 
findings that conflict with the commonly- 
accepted paradigm, that Rosenthal (1994) 
eloquently predicted would reside in the 



"file drawers" of researchers who could 
not get them published. In the case 
of ego-depletion, a unique contribution 
would be to identify unpublished stud- 
ies including those with null or negative 
effects, as well as studies that have since 
been published, and recalculate the meta- 
analytic effect size. Such an undertaking 
would not only yield a more robust effect 
size ostensibly independent of publica- 
tion bias but also be informative as to 
whether the "small study effect" detected 
in the analyses was due to publication 
bias, other forms of bias, or true het- 
erogeneity. We encourage researchers to 
make their replications of ego-depletion 
studies freely available to aid future 
meta-analyses. 

We would also like to express concerns 
regarding Carter and McCullough's pre- 
diction, based on their regression analyses, 
that the ego-depletion effect may be zero. 
This prediction was based on the inter- 
cept of the regression of the ego-depletion 
effect size on precision. However, if the 
true ego-depletion effect size is zero or 
close to it, one would expect the effect sizes 
in the literature to be randomly distributed 
in both positive and negative directions 
about zero. If this is the case, then where 
are those negative findings? There are scant 
few ego-depletion experiments that have 
found opposite effects, i.e., an improve- 
ment in second-task performance after 
engaging in an initial self-control task, let 
alone null effects. Given the intensiveness 
of research in this field, would it not be rea- 
sonable to expect to have seen the negative 
findings published? The absence of these 
effects creates a problem for the claim that 
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the true effect is zero and the credibility of 
the analysis. 

It could be argued that such nega- 
tive effects might not have been pub- 
lished because their interpretation might 
contradict commonly-accepted theory and 
may lie in the file drawers of the 
researchers who found them. However, 
we think that such findings would likely 
have seen the light of day in journals 
because they contradict the strength model 
and support alternative hypotheses consis- 
tent with other theories such as adapta- 
tion or learned industriousness (Converse 
and DeShon, 2009). For example, one 
could pose an alternative hypothesis that 
improvement in self-control performance 
in ego-depletion experiments could be due 
to learning the capacity to self-regulate 
which was transferable. In such cases 
one would expect statistically-significant 
improvements in performance on a sub- 
sequent self-control task after engaging in 
an initial task that taxes self-control. Of 
course, we would have to assume that 
researchers were sufficiently virtuous in 
not turning their null results into sup- 
portive evidence using selective reporting 
(Francis, 2014). We contend that if the pre- 
dicted effect size for ego-depletion is zero, 
then negative effects should be present in 
this literature and we would expect such 
effects to be published given their pivotal 
role in testing alternative hypotheses based 
on other theories. 

As a final point, while we thank Carter 
and McCullough for raising a notable 
question regarding the existence of bias 
in ego-depletion meta-analysis, their anal- 
ysis tells us little about its source and 
does not acknowledge that other effects 
in published meta-analyses are subject to 
similar bias. We think it is important to 
view and interpret the bias found for ego- 
depletion using these methods in context. 
For example, how does the small study 
effect found for ego depletion match up 
to the relative to the incidence of bias 
in the discipline of social psychology as 
whole? A useful future endeavor would 
be to systematically identify meta-analyses 



published in social psychology over a 
substantive period, subject each to the 
bias-identification analyses, and comment 
on the extent of the bias within the 
discipline. 
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Figure S1 | Funnel plot of the ego-depletion 
effect size (standardized mean difference, 
Cohen's d) against the against the study 
precision (1/standard error). 

Figure S2 | Funnel plot of the ego-depletion 
effect size (standardized mean difference, 
Cohen's d) against the standard error of the 
effect size. 
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